Image collection annotation using a mobile communicator

ABSTRACT

Disclosed herein are embodiments directed to a multi-media image system, which includes using a processor for forming a multi-media image product. According to certain embodiments, the system stores an image collection on a digital storage system; associates an identifier with the image collection: provides the identifier for distribution to a plurality of individuals; receives images from the recipients of the identifier; and forming a multi-media image product using the images stored on the digital storage system and received from the recipients of the identifier.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 13/074,403, filed on Mar. 29, 2011, entitled “ImageCollection Annotation Using a Mobile Communicator.” That application ishereby incorporated by reference in its entirety.

FIELD OF THE INVENTION

This invention pertains to the field of digital imaging and moreparticularly to methods for a group of people to provide content andannotations useful for forming a multi-media image product.

BACKGROUND OF THE INVENTION

Digital images are widely used to commemorate events in people's lives.Such digital images are typically captured by digital cameras, storedtemporarily in the digital cameras and then stored in electronic storagesystems such as home computers. The availability of internet-accessiblecomputers has also led to on-line digital image storage, for examplewith on-line data storage services, photo-merchandising services, socialnetworking websites, and photo-sharing websites.

Digital cameras and other mobile communication devices are widely used.These devices are capable of capturing images and communicating capturedimages wirelessly through a network to a computer storage device. Suchnetworks include local area computer networks and cellular telephonynetworks connected to computer networks. The computer networks are thentypically connected to the internet. Most mobile communication devicesprovide audio telephony services and many also provide digitalcommunication services such as text transmission or access to theinternet through a processor executing software on the mobilecommunication device. As used herein, a mobile communication deviceincludes portable devices that can interact remotely with other devices,typically through electromagnetic radiation such as cellular telephones,wireless computer networks, or optical readers or communicators.

On-line social network websites facilitated by the internet are popularfor online digital image storage. On these websites, users can uploadand store their digital images and provide access to the digital imagesto others in their social network, thereby sharing the digital imageswith the others. It is also known for others to record comments relevantto a digital image on the website.

Voice-recording and playback technology in computer systems is also wellestablished. Audio can be input and digitized by computers, for exampleusing a microphone with a USB connection and computer accessory hardwaresuch as an analog-to-digital convertor. The digitized audio file canthen be stored, for example as a file on a hard drive in a standardaudio file format. Similarly, software programs that execute on acomputer can read standard audio files and, with the use of adigital-to-analog convertor connected to a speaker, convert the digitalaudio file to sound. Voice messaging and telephone answering systems arealso known and widely used.

However, such mobile communication devices, internet-mediatedcommunication systems, and computer-based audio capabilities do notreadily provide a means for simple and efficient group input inproviding a collection of images related to an event and integratingcomments with the collection for use by an individual.

SUMMARY OF THE INVENTION

The present invention sets forth a method for forming a multi-mediaimage product, comprising:

using a processor to provide a storage location for a multi-media imagecollection in a digital storage system;

associating an identifier with the storage location, wherein theidentifier is adapted to be transmitted from or with a mobilecommunication device; providing the identifier for distribution to aplurality of individuals;

receiving a plurality of multi-media content from two or more of theplurality of individuals, the multi-media content being transmitted fromone or more mobile communication devices operated by the respectiveindividuals employing the identifier, wherein the multi-media contentincludes one or more images, audio annotations, or text annotationscaptured by the mobile communication device;

storing the multi-media content in the storage location and associatingthe multi-media content with the multi-media collection; providing rulesfor associating the multi-media content with the images or themulti-media image collection; and making a multi-media image productusing at least a portion of the multi-media content stored in themulti-media collection wherein the multi-media content is associatedwith the images or multi-media image collection in accordance with therules.

This invention has the advantage that it provides for audio annotationwhen the provision of text is inconvenient, especially for mobiledevices that often have limited interactive functionality. It has theadditional advantage of enabling collaboration for securely making animage collection and of enabling simple and convenient interaction forparticipants.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level diagram showing the components of a system forproviding content and annotations according to an embodiment of thepresent invention;

FIG. 2 is schematic of a computer system useful in the presentinvention;

FIG. 3 is schematic of a computer system useful in the presentinvention;

FIG. 4 is a view of a mobile communication device useful in the presentinvention; and

FIG. 5 is a flow diagram illustrating various embodiments of the presentinvention.

It is to be understood that the attached drawings are for purposes ofillustrating the concepts of the invention and may not be to scale.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, some embodiments of the present inventionwill be described in terms that would ordinarily be implemented assoftware programs. Those skilled in the art will readily recognize thatthe equivalent of such software can also be constructed in hardware.Because image manipulation algorithms and systems are well known, thepresent description will be directed in particular to algorithms andsystems forming part of, or cooperating more directly with, the methodin accordance with the present invention. Other aspects of suchalgorithms and systems, together with hardware and software forproducing and otherwise processing the image signals involved therewith,not specifically shown or described herein can also be selected fromsuch systems, algorithms, components, and elements known in the art.Given the system as described according to the invention in thefollowing, software not specifically shown, suggested, or describedherein that is useful for implementation of the invention isconventional and within the ordinary skill in such arts.

The invention is inclusive of combinations of the embodiments describedherein. References to “a particular embodiment” and the like refer tofeatures that are present in at least one embodiment of the invention.Separate references to “an embodiment” or “particular embodiments” orthe like do not necessarily refer to the same embodiment or embodiments;however, such embodiments are not mutually exclusive, unless soindicated or as are readily apparent to one skilled in the art. The useof singular or plural in referring to the “method” or “methods” and thelike is not limiting. It should be noted that, unless otherwiseexplicitly noted or required by context, the word “or” is used in thisdisclosure in a non-exclusive sense.

The phrase, “digital image file”, as used herein, refers to any digitalimage file, such as a digital still image or a digital video file.

FIG. 1 is a high-level diagram showing the components of a system forannotation of an image collection according to an embodiment of thepresent invention. The system includes a data processing system 110, aperipheral system 120, a user interface system 130, and a data storagesystem 140. The peripheral system 120, the user interface system 130andthe data storage system 140 are communicatively connected to the dataprocessing system 110.

The data processing system 110 includes one or more data processingdevices that implement the processes of the various embodiments of thepresent invention, including the example processes described herein. Thephrases “data processing device” or “data processor” are intended toinclude any data processing device, such as a central processing unit(“CPU”), a desktop computer, a laptop computer, a mainframe computer, apersonal digital assistant, a Blackberry™, a digital camera, a digitalpicture frame, cellular phone, or any other device for processing data,managing data, or handling data, whether implemented with electrical,magnetic, optical, biological components, or otherwise.

The data storage system 140 includes one or more processor-accessiblememories configured to store information, including the informationneeded to execute the processes of the various embodiments of thepresent invention, including the example processes described herein. Thedata storage system 140 can be a distributed processor-accessible memorysystem including multiple processor-accessible memories communicativelyconnected to the data processing system 110 via a plurality of computersor devices. On the other hand, the data storage system 140 need not be adistributed processor-accessible memory system and, consequently, caninclude one or more processor-accessible memories located within asingle data processor or device.

The phrase “processor-accessible memory” is intended to include anyprocessor-accessible data storage device, whether volatile ornonvolatile, electronic, magnetic, optical, or otherwise, including butnot limited to, registers, caches, floppy disks, hard disks, CompactDiscs, DVDs, flash memories, ROMs, and RAMs.

The phrase “communicatively connected” is intended to include any typeof connection, whether wired or wireless, between devices, dataprocessors, or programs in which data can be communicated. The phrase“communicatively connected” is intended to include a connection betweendevices or programs within a single data processor, a connection betweendevices or programs located in different data processors, and aconnection between devices not located in data processors at all. Inthis regard, although the data storage system 140 is shown separatelyfrom the data processing system 110, one skilled in the art willappreciate that the data storage system 140 can be stored completely orpartially within the data processing system 110. Further in this regard,although the peripheral system 120 and the user interface system 130 areshown separately from the data processing system 110, one skilled in theart will appreciate that one or both of such systems can be storedcompletely or partially within the data processing system 110.

The peripheral system 120 can include one or more devices configured toprovide digital content records to the data processing system 110. Forexample, the peripheral system 120 can include digital still cameras,digital video cameras, cellular phones, or other data processors. Thedata processing system 110, upon receipt of digital content records froma device in the peripheral system 120, can store such digital contentrecords in the data storage system 140.

The user interface system 130 can include a mouse, a keyboard, anothercomputer, or any device or combination of devices from which data isinput to the data processing system 110. In this regard, although theperipheral system 120 is shown separately from the user interface system130, the peripheral system 120 may be included as part of the userinterface system 130.

The user interface system 130 also can include a display device, aprocessor-accessible memory, or any device or combination of devices towhich data is output by the data processing system 110. In this regard,if the user interface system 130 includes a processor-accessible memory,such memory can be part of the data storage system 140 even though theuser interface system 130 and the data storage system 140 are shownseparately in FIG. 1.

FIG. 2 illustrates a more detailed embodiment of an electronic system20, for example a computer, useful for enabling the present invention.In the embodiment of FIG. 2, an electronic computer system 20 includes asource of content data files 24, a user input system 26, and an outputsystem 28 connected to a processor 34. The source of content data files24, user-input system 26 or output system 28 and processor 34 can bepositioned within a housing 22 as illustrated.

In other embodiments, circuits and systems of the source of content datafiles 24, user input system 26 or output system 28 can be positioned inwhole or in part outside of housing 22.

The source of content data files 24 can include any form of electronicor other circuit or system that can supply digital data to processor 34from which processor 34 can derive images for use in forming animage-enhanced item. In this regard, the content data files can include,for example and without limitation, still images, image sequences, videographics, and computer-generated images. Source of content data files 24can optionally capture images to create content data for use in contentdata files by use of capture devices located at, or connected to,electronic computer system 20 or can obtain content data files that havebeen prepared by or using other devices such as a remote electroniccomputer system 35. In the embodiment of FIG. 2, source of content datafiles 24 includes sensors 38, a memory 40 and a communication system 54.

Sensors 38 are optional and can include light sensors, biometric sensorsand other sensors known in the art that can be used to detect conditionsin the environment of electronic computer system 20 and to convert thisinformation into a form that can be used by processor 34 of electroniccomputer system 20. Sensors 38 can also include one or more imagesensors 39 that are adapted to capture still or video images. Sensors 38can also include biometric or other sensors for measuring involuntaryphysical and mental reactions, such sensors 38 including, but notlimited to, voice inflection, body movement, eye movement, pupildilation, body temperature, and p4000 wave sensors.

Memory 40 can include conventional memory devices including solid-state,magnetic, optical or other data-storage devices. Memory 40 can be fixedwithin electronic computer system 20 or it can be removable. In theembodiment of FIG. 2, electronic computer system 20 is shown having ahard drive 42, a disk drive 44 for a removable disk such as an optical,magnetic or other disk memory (not shown) and a memory card slot 46 thatholds a removable memory 48 such as a removable memory card and has aremovable memory interface 50 for communicating with removable memory48. Data including, but not limited to, control programs, digital imagesand metadata can also be stored in a remote memory system 52 such as anelectronic computer system 35, a personal computer, computer network orother digital system. Remote memory system 52 can also includesolid-state, magnetic, optical or other data-storage devices.

In the embodiment shown in FIG. 2, electronic computer system 20 has acommunication system 54 that in this embodiment can be used tocommunicate with remote electronic computer systems 35 for exampleincluding an optional remote memory system 52, an optional remotedisplay 56, or optional remote input 58 (also referred to herein as“remote input 58”). The optional remote memory system 52, optionalremote display 56, and optional remote input 58 can all be part of aremote electronic computer system 35 having an input station with remoteinput controls, and that can communicate with communication system 54wirelessly as illustrated or can communicate in a wired fashion. In analternative embodiment, a local input station including either or bothof a local display 66 and local input controls 68 (also referred toherein as “local user input 68”) can be connected to communicationsystem 54 using a wired or wireless connection.

Communication system 54 can comprise for example, one or more optical,radio frequency or other transducer circuits or other systems thatconvert image and other data into a form that can be conveyed to aremote device such as remote memory system 52 or remote display 56 usingan optical signal, radio frequency signal or other form of signal.Communication system 54 can also be used to receive a digital image andother data from a host or server computer or network (not shown), aremote memory system 52 or a remote input 58. Communication system 54provides processor 34 with information and instructions from signalsreceived thereby. Typically, communication system 54 will be adapted tocommunicate with the remote memory system 52 by way of a communicationnetwork such as a conventional telecommunication or data transfernetwork such as the internet, a cellular, peer-to-peer or other form ofmobile telecommunication network, a local communication network such aswired or wireless local area network or any other conventional wired orwireless data transfer system. In one useful embodiment, the electroniccomputer system 20 can provide web access services to remotely connectedelectronic computer systems 35 that access the electronic computersystem 20 through a web browser. Alternatively, a remote electroniccomputer system 35 can provide web services to electronic computersystem 20 depending on the configurations of the systems.

User input system 26 provides a way for a user of electronic computersystem 20 to provide instructions to processor 34. This permits such auser 72 (FIG. 3) to make a designation of content data files to be usedin generating an image-enhanced output product and to select an outputform for the output product. User input system 26 can also be used for avariety of other purposes including, but not limited to, permitting auser 72 (FIG. 3) to arrange, organize and edit content data files to beincorporated into the image-enhanced output product, to provideinformation about the user 72 (FIG. 3) or audience, to provideannotation data such as voice and text data, to identify characters inthe content data files, and to perform such other interactions withelectronic computer system 20 as are described herein.

In this regard, user input system 26 can comprise any form of transduceror other device capable of receiving an input from a user 72 (FIG. 3)and converting this input into a form that can be used by processor 34.For example, user input system 26 can comprise a touch screen input, atouch pad input, a 4-way switch, a 6-way switch, an 8-way switch, astylus system, a trackball system, a joystick system, a voicerecognition system, a gesture recognition system a keyboard, a remotecontrol or other such systems. In the embodiment shown in FIG. 2, userinput system 26 includes an optional remote input 58 including a remotekeyboard 58 a, a remote mouse 58 b, and a remote control 58 c and alocal input 68 including a local keyboard 68 a and a local mouse 68 b.

Remote input 58 can take a variety of forms, including, but not limitedto, the remote keyboard 58 a, remote mouse 58 b or remote controlhandheld device 58 c illustrated in FIG. 2. Similarly, local input 68can take a variety of forms. In the embodiment of FIG. 2, local display66 and local user input 68 are shown directly connected to processor 34.

As is illustrated in FIG. 3 local user input 68 can take the form of ahome computer, an editing studio, or kiosk 70 (hereafter also referredto as an “editing area 70”) that can also be a remote electronic orelectronic computer system 35. In this illustration, a user 72 is seatedbefore a console comprising local keyboard 68 a and mouse 68 b and alocal display 66 which is capable, for example, of displaying multimediacontent, for example in a graphic user interface. As is also illustratedin FIG. 3, editing area 70 can also have sensors 38 including, but notlimited to, image sensors 39, audio sensors 74 and other sensors such asmultispectral sensors that can monitor user 72 during a productionsession or provide other information such as images. Storage devicessuch as disk drive 44 are included in electronic computer system 35.

Output system 28 (FIG. 2) is used for rendering images, text or othergraphical representations in a manner that permits image-product designsto be combined with user items and converted into an image product. Inthis regard, output system 28 can comprise any conventional structure orsystem that is known for printing or recording images, including, butnot limited to, printer 29. Printer 29 can record images on a tangiblesurface 30 using a variety of known technologies including, but notlimited to, conventional four-color offset separation printing or othercontact printing, silk screening, dry electro-photography such as isused in the NexPress 2100 printer sold by Eastman Kodak Company,Rochester, N.Y., USA, thermal printing technology, drop-on-demand inkjettechnology and continuous inkjet technology. For the purpose of thefollowing discussions, printer 29 will be described as being of a typethat generates color images. However, it will be appreciated that thisis not required and that the claimed methods and apparatuses herein canbe practiced with a printer 29 that prints monotone images such as blackand white, grayscale, or sepia toned images. As will be readilyunderstood by those skilled in the art, electronic computer systems 35,20 with which a user 72 interacts to define a user-personalized imageproduct can be separated from a remote computer system (e.g. 35)connected to a printer 29 or other output device 32, so that thespecification of the image product is remote from its production

In certain embodiments, the source of content data files 24, user inputsystem 26 and output system 28 can share components.

Processor 34 operates electronic computer system 20 based upon signalsfrom user input system 26, sensors 38, memory 40 and communicationsystem 54. Processor 34 can include, but is not limited to, aprogrammable digital computer, a programmable microprocessor, aprogrammable logic processor, a series of electronic circuits, a seriesof electronic circuits reduced to the form of an integrated circuit, ora series of discrete components.

In a useful embodiment of the present invention illustrated in FIG. 4,the remote electronic computer system 35 of FIG. 2 can be a mobilecommunication device 80, for example a cellular telephone, a smartphone, or a wirelessly connected hand-held computer such as a tabletcomputer. The mobile communication device 80 can include a local display66 or a user-input system 26 (FIG. 2) incorporating a local input 68keyboard 68 a. In another embodiment, a local input 68 device can be atouch screen. The mobile communication device 80 can communicate with anelectronic computer system 20 (e.g. as shown in FIG. 2) directly, forexample through a wireless local area network or a point-to-pointwireless connection, or indirectly through a cellular telephony network.The electronic computer system 20 can be a computer server, for exampleproviding browser-based web pages to a remote electronic computer system35.

Any of the server computer 20, remote electronic computer system 35, orthe mobile communication device 80 can execute software programs on aninternal processor 34. The software programs can interact with the user72 through a user interface (e.g. local display 66 and local input 68)or with remote computers to accomplish the programmed task. The softwareprograms can execute algorithms to analyze data (e.g. digital imagefiles) or to compute useful values. A computer software program productcan include one or more non-transitory, tangible, computer readablestorage medium, for example; magnetic storage media such as magneticdisk (such as a floppy disk) or magnetic tape;

optical storage media such as optical disk, optical tape, or machinereadable bar code; solid-state electronic storage devices such as randomaccess memory (RAM), or read-only memory (ROM); or any other physicaldevice or media employed to store a computer program having instructionsfor controlling one or more computers to practice the method accordingto the present invention.

The present invention will now be described with reference to FIG. 5.According to an embodiment of the present invention, a method forannotation of an image collection includes using a processor (e.g.electronic computer system 20) to provide a storage location for animage collection in a digital storage system (e.g. disk drive 44 orremote memory system 52) in step 200. This can be accomplished byproducing a storage location (e.g. a file folder) in acomputer-controlled memory system 52 which can be remotely accessible,for example through the internet. The digital storage system (e.g. diskdrive 44 or remote memory system 52) can be controlled or owned by athird party other than the one that enables the image collection. Forexample, a user 72 can specify a folder for storing digital images in adigital storage system on a computer or specify an album on aphoto-sharing website on a remote memory system 52. The user 72 can alsostore digital images within the collection, if desired, by accessing thecomputer controlling the storage location and copying images into theimage collection or uploading images to a computer server that controlsthe storage location. As used herein, an image collection is a set ofimages having a common logical storage location, although the data forthe images in the collection can be distributed over multiple storagedevices such as disk drives, as is well known in the computer storageindustry. The term image collection is used similarly to the termstorage location, and generally refers to the logical storage locationof the images in the collection. The image collection can have no imagesstored in the storage location when it is first specified, for exampleas is the case when a file folder is first defined in an operatingsystem.

In step 205, an identifier is associated with the storage location ofthe image collection. The identifier can also be associated with theimage collection. In various embodiments, the identifier can be selectedby the user 72, for example as an album name, or can be provided by theowner of the storage system, if different, for example as an encodedtext string. Generally, it is preferred that the identifier be unique,so that if the user 72 selects the identifier, additionalcharacteristics or codes can be added to the identifier. The identifiercan be a name, an email address, or a telephone number and can includeadditional values or codes. The identifier serves to enable storingdata, such as images, in the image collection at the storage location orto access stored data in the image collection at the storage location.The identifier can be an address in a computer system, network, orstorage system or it can refer to such an address. If the identifierrefers to an address or location, a table associating identifiers withspecific storage locations can be provided, for example by a table thatassociates identifiers with addresses. Such tables and data associationsare well known in the programming arts.

The identifier is then provided for distribution to a plurality ofindividuals in step 210. This distribution can be done in a variety ofways according to various embodiments. For example, the identifier canbe emailed to the individuals, provided verbally, or printed anddistributed as a hard copy notice. The identifier can be provideddirectly or indirectly by the user 72, through the electronic computersystem 20, or under the control of electronic computer system 20directly to the plurality of individuals or for distribution by the user72 or others. Individuals that receive the identifier can also furtherdistribute the identifier to additional individuals that can have aninterest in the image collection. One or more of the plurality ofindividuals receives the identifier in step 211.

In step 214, one or more of the individuals sends the identifier withone or more images to, for example, a computer server on the internet(e.g. electronic computer system 20). In step 215, informationassociated with the identifier or the identifier and one or more imagesare received from a first individual and the received images, forexample digital images, are associated with the image collection andstored on the storage device (e.g. hard drive 42) at the storagelocation. The first individual can be a person to whom the identifierwas distributed. The identifier is sent by the first individual to theelectronic computer system 20 to identify the image collection withwhich received images are to be associated. Alternatively, thearrangement by which the images are sent or addresses to which they aresent constitute the identifier or information associated with theidentifier.

The identifier and images can be transmitted and received in a varietyof ways, for example by email, by surface mail (for example with imagesstored on a storage medium such as a compact disk), or through atelephonic connection (e.g. a data transmission through a landlinetelephone, cell phone or smart phone). If an electronic transmission isused, the address of the receiver can be the identifier, or a portion ofthe identifier, for example an internet URL or a telephone number. Inthis case the access address is not necessarily received but is ratherthe mechanism by which an electronic contact is made. The electroniccomputer system 20 can then associate the address with a specificstorage location and image collection. The association constitutesadditional information associated with the identifier. For example, anemail address or telephone number can be an identifier associated withan image collection and storage location. When information is receivedby the electronic computer system 20 at that electronic address, E-mailaddress, or telephone number, the electronic computer system 20 canemploy a look up table or other electronic reference stored in a memoryor register to discover the image collection and storage locationassociated with the electronic address, E-mail address, or telephonenumber.

In various non-limiting embodiments, the identifier can be an emailaddress, a URL, a URI, a text string, a graphic code such as a bar codeor QR code, or a telephone number. Additional codes can be employed withthe identifier or as part of the identifier to further identify thetransmitted and received images and their associated image collection.E-mail addresses, URLs, or telephone numbers are explicitly included asvarious embodiments of the present invention, together with any otheridentifying codes. For example, a user might call a specific telephonenumber or send E-mail to a specific address and then provide furtherinformation (e.g. a number, text string, or name) to associatetransmitted information with the desired image collection and storagelocation. Electronic equipment for receiving codes entered from atelephone keypad or keyboard for telephones, computers, or web sites arewell known, for example using voice mail systems, electronic messaging,and web browsers.

In step 219, one or more of the individuals sends the identifier withone or more annotations to, for example, a computer server on theinternet. In step 220, information associated with the identifier or theidentifier and one or more annotations are received from the sameindividual or a second individual different from the first individualand the received annotations, for example audio or text annotations, areassociated with the digital images in the image collection and stored ona storage device (e.g. hard drive 42) Annotations can include, forexample, messages, observations, or comments relevant to a particularimage or to an entire image collection and can have a variety of formatsor forms, or constituted in various media. The identifier, or theinformation associated with the identifier can be similar to that ofstep 215 discussed above and the communication method can be the same,or different. In addition to the first and second individuals, otheradditional individuals who have received the identifier can provideimages, multi-media content, or audio or textual annotations that arethen associated with the image collection and used in a multi-mediaproduct.

The annotations can be associated with specific images in the imagecollection either through the identifier alone, through codes includedwith the identifier, or through additional information received afterthe identifier is sent, for example by asking a user to enter additionalinformation identifying the images to be associated with theannotations, for example by using a voice recognition system to selectimages or through text entries (e.g. a number known to be associatedwith a specific image, images, or the image collection).

In step 225, rules are provided that associate annotations with specificimages or with the image collection in general. In various embodiments,multiple annotations can be associated with a single image, or a singleannotation can be associated with multiple images. One or moreannotations can be associated with the image collection only or with allimages in the image collection. Furthermore, rules specifying thepresentation of annotations (either text or audio) with respect to anyassociated images can be used to specify the manner in which theannotations and images are presented in the multi-media productAnnotations from different sources or contributors can be widelydistributed in the collection or product, limited, grouped, orseparated. For example, it can be desired to limit the number or lengthof annotations from a single contributor or the content of an annotationto ensure that the content is suitable to the collection (e.g. limitinginappropriate language or image content).

Rules can be provided that customize the annotations, images, imagecollection, or multi-media product for different recipients of themulti-media product. The rules can also specify the nature of themulti-media product in response to the types of annotations and digitalimages.

For example, in a hard-copy multi-media product, rules can specifytextual annotations printed in different locations in association withan image or in locations not associated with any images but with theimage collection. Audio annotations can be played when evoked, forexample by viewing an image or image sequence, or by activating a switchlocated in different locations in association with an image or withmultiple images, or in a location not associated with a specific imageor group of images. Rules governing multiple audio annotationsassociated with an image or with the image collection can specify thatthey are all played sequentially when activated with a single switchactivation, or that a separate activation can be required to play eachaudio annotation, or that a random selection of one or more audioannotations can be made in response to a switch activation.

A digital memory can be provided for storing the audio annotation, rulesfor associating images and collections, and a control for activatingcircuits to play the audio annotation from the digital memory inaccordance with rules.

In a digital multi-media product, rules specify that textual annotationscan be displayed in different locations in association with an image orin locations not associated with any images but with the imagecollection. Audio annotations can be played when evoked by selecting acontrol in all of the arrangements described above, or can be played inassociation with text annotations, or the text annotations can beaurally played using a text-to-voice convertor. One skilled in thesoftware arts can store rules and implement the rules and functions witha processor and known hardware peripherals. A local display 66 canprovide a user interface in a computer system for presenting one or moreof the images in the image collection to a viewer or for presentingmulti-media content or annotations.

Default rules can be employed to reduce the number of choices that auser makes. The default rules can be supplied by a third party, one ofthe individuals, or by a vendor of the multi-media product.

Once the images, annotations, and rules are received and properlyassociated with the image collection, a multi-media productincorporating one or more of the digital images and annotations can bespecified. In an optional step 229, indicated by the dashed arrows inFIG. 5, the annotations, associations, rules and images can be edited asdesired by a recipient of a specified multi-media product or by a personorganizing or contributing to the specified multi-media product. Theediting can be done by one or more person, iteratively, before thespecification of the product (e.g. by editing images or annotations), orafter, to customize the multi-media product to different individuals orto the specific desires of one individual.

The annotations can include both text and audio information. A typicalperson can readily skim through printed text to read with attention onlythose portions of the text which are of greater interest to the readerwhile it is difficult to skim through an audio presentation. Hence, inan example, fewer audio annotations are associated with a singlecontrol, so that a user need not listen to as many annotations beforemoving to a new element in the multi-media product. Furthermore, rulescan be specific to the media, for example the font or type size of textis not a relevant attribute to an audio annotation, as volume in anaudio annotation is not relevant to a text annotation. Therefore,according to an embodiment, different rules can be provided fordifferent types of annotation, for example text and audio.

Once the multi-media product is acceptable and completely specified, itcan be made in accordance with the rules (step 230). The multi-mediaproduct can be a hard-copy product (e.g. a photo-book or photo-collagewith textual annotation or audio playback) or a digital product (forexample a digital image slideshow, digital photo album, or digitalcollage with audio that is played back with a software-controlled mediaplayer. The multi-media image product can be a digital multi-media imageproduct stored in a digital storage medium.

The multi-media image product can be a hard-copy multi-media imageproduct with an associated link to a playback device. For example,non-electronic image products such as mugs can include printed codesthat include information that, when scanned with an electronic scanner(e.g. a smart phone), enable audio playback either by encoding an audiofile or by encoding a reference to an audio file, such as a URL. Anexample of a useful code is a one-dimensional or two-dimensional barcode such as a QR code. Such playback or remote access to electronicallystored information from printed codes is known in the electronic andcomputer arts.

In either the hard-copy case or digital (soft-copy) case, themulti-media product can be delivered to customers (step 235), forexample either by surface mail (for hard-copy products) or by E-mail ormade accessible from an on-line digital storage repository (for digitalproducts).

In one embodiment of the present invention, the annotations are audioannotations. An audio annotation can be an electronic audio file, avoice recording provided by an individual, or a selection of apre-recorded audio message. An audio annotation can be received as anE-mail, an audio file recorded on a physical medium (e.g. an opticaldisc), by recording a telephonic message into an audio file, through aweb browser executing on a computer, or from a software applicationexecuting on a processor. The audio annotation can be received from asoftware application installed on a client computer that interacts witha remote server connected to the client computer through a computernetwork (e.g. as shown in FIGS. 2 and 3). Alternatively, the audioannotation can be received from a web browser executing on a clientcomputer that presents information provided from a remote serverconnected to the client computer through a computer network.

For example, the identifier can be a telephone number and a customer cancall the telephone number and make an audio recording that is recordedby an audio recording device accessed through the telephone, forexample, by using technologies such as are used for voice mail, toprovide an annotation. Previously made audio recordings can also bereviewed or edited. In an alternative example, the identifier can be anemail address and a customer can record a digital audio file (forexample using digital audio tools such as USB microphones connected tocomputers) and email the audio file to the email address.

In another embodiment of the present invention, an identifier, images,or annotations are received from a mobile cellular telephone. Theannotations can take a variety of forms, for example the annotations canbe either textual or audio. Contemporary cellular telephones (e.g.mobile communication device 80) can incorporate user interfaces thatinclude text input (for example a keypad) or audio input (using themicrophone for inputting sound to make an electronic audio signal). Theuser interface can be used to make an image annotation that is thentransmitted to an electronic computer system 20 through the cellulartelephone network. Other communications can also be used, for examplepoint-to-point electromagnetic communication (e.g. Bluetooth) or localarea networks, either wired or wireless.

Therefore, an alternative method of the present invention includes usinga processor to provide a storage location for a multi-media imagecollection on a digital storage system and associating an identifierwith the storage location and image collection. The identifier isadapted to be communicated from or with a mobile communication device,for example through a cellular telephone making a telephone call. Theidentifier is provided for distribution to a plurality of individuals.Two or more of the individuals, or others who have been provided withthe identifier, transmit a plurality of multi-media content employingthe identifier to the processor from one or more mobile communicationdevices operated by the respective individuals, for example mobilecommunication devices. The multi-media content includes one or moreimages, audio annotations, or text annotations captured by the mobilecommunication devices, for example cellular telephones. The multi-mediacan include the identifier or the identifier can be an address (e.g.telephone number, E-mail address, or network location address) with orwithout additional identifying information. The received multi-mediacontent is stored in the storage location and associated with themulti-media collection. Rules are provided for associating themulti-media content with the images or the multi-media image collection.The resulting specification of the multi-media product can be edited asdesired. A multi-media image product using at least a portion of themulti-media content stored in the multi-media collection is made,wherein the multi-media content is associated with the images ormulti-media image collection in accordance with the rules.

Referring to FIGS. 2 and 3, for example, a system for annotation of animage collection can include a digital storage system for storing animage collection in a digital storage system, the image collectionhaving an associated identifier. The image collection can be amulti-media image collection, for example the images can be still,video, include audio, textual information, or other content. The systemcan include communication ways for distributing the identifier to aplurality of individuals, for example by mail, E-mail, printedinformation, or verbally and ways for receiving information associatedwith the identifier or the identifier together with one or more imagesfrom a first individual. Alternatively, the information can be receivedfrom a plurality of individuals, for example using mobile communicationdevices 80, such as cellular telephones. The communication ways includesassociating the received image(s) with the storage location, for exampleby using a processor to use the identifier to locate the storagelocation and a computer-controlled file system to associate the receivedimages with the multi-media image collection, for example using alogical file folder structure. The system also includes communicationways for receiving information associated with the identifier or theidentifier together with multi-media content such as one or more text oraudio annotations from a plurality of individuals or a second individualdifferent from the first individual and associating the receivedmulti-media content, text annotation(s), or audio annotation(s) with thestorage location, for example by using a processor to store themulti-media content, text annotation(s), or audio annotation(s) in alogical storage structure such as a file folder. The system can alsoprovide rules for associating multi-media content, text annotations, oraudio annotations with the images or the image collection. The rules canbe, for example, provided by a multi-media product vendor or specifiedby a user using an interactive software application or through aweb-based application on a remote computer or mobile communicationdevice. The system can include a digital memory for storing audio ortext annotations, a processor for storing and applying rules forassociating images, annotations, and collections, and a control foractivating circuits to play the annotations from the digital memory inaccordance with rules. The system can also include ways for making amulti-media image product that includes one or more images from theimage collection together with multi-media content, one or more textannotations, or one or more audio annotations wherein the multi-mediacontent, one or more text annotations, or one or more audio annotationsare associated with the images or image collection in accordance withthe rules. Such ways can include controlling manufacturing equipment orcommunicating a multi-media product specification to a separate system,for example a manufacturing system owned by a third party. The systemcan also provide ways for editing the specification of the multi-mediaproduct, for example by modifying the rules, annotations, associations,or images in the image collection.

In yet another embodiment of the present invention, a computer programproduct for audio annotation of an image collection uses the methodsdescribed above and can execute on a computer system such as thatdescribed above.

In various embodiments of the present invention, the identifier is auniversal resource locater (URL), a universal resource identifier (URI),a telephone number, an identification number, a collection name, anelectronic code, an optically recognizable code, an instant messageaddress, or an E-mail address. An optically recognizable code can be aone-dimensional bar code or a two-dimensional bar code, such as a QRcode. Such codes contain information and are known in the prior art. Forexample, a user can scan an optically recognizable code and employ asoftware application to direct images or multi-media content to alocation specified in the code. The first individual can be anindividual who established the image collection. Alternatively, a thirdindividual different from the first and second individuals can be anindividual who establishes the image collection. Various differentindividuals can also specify the formation of the multi-media imageproduct, for example any of the individuals that contribute images orannotations to the image collection or establish the image collection,or others. An individual who specifies the image collection can alsoinitialize the image collection by providing an initial set of imagesthat are stored in the storage location in association with the imagecollection, for example in a file folder or album collection. The imagecollection can thus contain images from a variety of individuals and themulti-media image product can likewise contain images from differentindividuals, such as the individual that initialized the imagecollection and an individual that later added images or othermulti-media content or annotations in association with the imagecollection.

The present invention is useful for facilitating the production ofmulti-media image products based on collaboration at social occasions.For example, weddings are popular events for photography. Anotherexample is funerals at which family members gather to commemorate adeceased relative. In one non-limiting embodiment of the presentinvention, a photographer provides a storage location for an imagecollection in an on-line album or repository on an internet-accessiblecomputer server. The storage location can be specific to a particularsocial occasion, for example a wedding and can have an identifier,chosen by the photographer or by the computer server associated with thestorage location, for example an internet address with a suffixincluding an occasion name. In another embodiment, the identifier is atelephone number with a numeric or alphabetic code relevant to theoccasion name. The photographer can make digital photographs of, orrelevant to, the social occasion and store the photographs in thestorage location and can also distribute the identifier to guests at thesocial occasion. The photographs can include recent or recent historicalphotographs.

The guests can use the identifier to access the stored photographs, forexample viewing them in a browser on a computer, for example a desktopcomputer or a mobile communication device with a display and havinginternet access for viewing remotely stored content, such as a smartphone. Guests can then record comments relevant to the social occasionor the stored photographs and upload them to the storage location toassociate the comments with the image collection. Guests can also uploaddigital images to the repository, for example images taken at the socialoccasion or relevant images of historical interest possessed by one ormore of the guests. These uploaded images can also be viewed by otherguests and further comments or annotations provided. In yet anotherembodiment, guests can upload pre-recorded audio clips that express adesired sentiment, for example selected from a library of such audioclips.

At the conclusion of the social occasion, or after some designatedperiod after, a multi-media image product that includes one or moreimages from the image collection together with one or more of theannotations is made and can be provided to attendees or guests of honorat the social occasion.

The multi-media product can be made in association with rules for howthe product is to operate. Such rules can be decided by a vendor of theproduct or by a professional photographer that facilitates theconstruction of the multi-media image collection, or by a professionalphotographer in consultation with an organizer of the social occasion orguest at the social occasion.

In one embodiment of the present invention, the photographer is awedding photographer hired by a wedding organizer and guests at thewedding provide annotations and images to an image collection initiatedby the wedding photographer. Guests can make images and provideannotations at a reception or thereafter, and upload them when made orlater. The photographer can then organize, specify, and construct (orhave a third party construct) a multi-media product commemorating theoccasion according to the rules. The photographer can review thespecified multi-media product and edit it as desired, or consult withthose that desire to receive the organized multi-media product andpermit them to edit the specification or edit the specification inaccordance with their desires.

In another embodiment of the present invention, a funeral organizerhired by a family member, a friend, or a family member organizes asocial occasion to mourn or celebrate the life of deceased friend orfamily member. The organizer instantiates the image collection andguests at the occasion provide images or annotations to the imagecollection either at the time, before, or after. At the conclusion ofthe event, or sometime thereafter, the organizer can specify andconstruct (or have a third party construct) a multi-media productcommemorating the occasion. Additionally, the specified product can beedited as described above.

In yet another embodiment of the present invention, a friend or familymember organizes the production of a memento such as a multi-mediaexpression of concern or support for an injured or ill friend,colleague, or relative. The organizer instantiates the image collectionand other members of the social community provide images or annotationsto the image collection. The images and annotations are used to producethe multi-media memento which is then presented to the injured or illfriend, colleague, or relative.

Other social occasions in addition to these described above can provideopportunities for the collaborative specification and subsequentconstruction and distribution of an annotated multi-media product andare included in the present invention.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention.

PARTS LIST

-   20 electronic computer system, server computer-   22 housing-   24 source of content data files-   26 user input system-   28 output system-   29 printer-   30 tangible surface-   32 output device-   34 processor-   35 electronic computer system-   38 sensors-   39 image sensors-   40 memory-   42 hard drive-   44 disk drive-   46 memory card slot-   48 removable memory-   50 memory interface-   52 remote memory system-   54 communication system-   56 remote display-   58 remote input-   58 a remote keyboard-   58 b remote mouse-   58 c remote control-   66 local display-   68 local input-   68 a local keyboard-   68 b local mouse-   70 home computer, editing studio, or kiosk (editing area)-   72 user-   74 audio sensors-   80 mobile communication device-   110 data processing system-   120 peripheral system-   130 user interface system-   140 data storage system-   200 enable image collection step-   205 associate identifier step-   210 distribute identifier step-   211 receive identifier step-   214 send identifier and images step-   215 receive identifier and images step-   219 send identifier and annotations step-   220 receive identifier and annotations step-   225 provide rules step-   229 optional edit step-   230 make product step-   235 deliver product step

1. A method of making a multi-media image product using a multi-mediaimage system comprising a data processing system and a data storagesystem, the method comprising: storing an image collection on the datastorage system, the image collection comprising a plurality of digitalimages; storing preset vendor rules on the data storage system, thevendor rules specifying a digital image layout and an annotationpresentation layout for a multi-media image product, wherein theannotation presentation layout specifies a first presentation format fortext annotations when the multi-media image product is a hard-copyproduct and specifies a second presentation format for audio annotationswhen the multi-media image product is a digital product; generating anidentifier associated with the image collection; distributing theidentifier electronically via a computer network to a user's remotedevice; receiving the identifier from the user via the computer network;granting the user remote access to the image collection via the computernetwork; receiving, from the user via the computer network, anannotation and an annotation assignment instruction, wherein theannotation assignment instruction instructs the annotation to beassociated with a specific digital image in the image collection, andwherein the annotation is a text annotation or an audio annotation;making the multi-media image product according to the preset vendorrules, wherein the multi-media image product includes the annotationreceived from the user.
 2. The method of claim 1, further comprisingediting the annotation received from the user.
 3. The method of claim 1,wherein the annotation is an audio annotation and the multi-media imageproduct is a digital product.
 4. The method of claim 1, furthercomprising: generating a code associated with the image collection;distributing the code with the identifier to the user's remote devicevia the computer network; receiving the code with the identifier fromthe user via the computer network; and granting the user remote accessto the image collection via the computer network upon receipt of boththe code and the identifier.
 5. The method of claim 1, furthercomprising: receiving, from the user via the computer network, an image;storing the image from the user on the data storage system; andincorporating the image from the user into the multi-media imageproduct.
 6. The method of claim 5, wherein the annotation received fromthe user is associated with the image received from the user.
 7. Themethod of claim 1, wherein the annotation presentation layout specifiesa font size for the text annotations when the multi-media image productis a hard-copy product.
 8. The method of claim 1, wherein the annotationpresentation layout specifies an audio volume for the audio annotationswhen the multi-media image product is a digital product.
 9. The methodof claim 1 further comprising: receiving a plurality of annotations froma plurality of users via the computer network; and storing the pluralityof annotations on the data storage system, wherein each one of theplurality of annotations is stored in association with a specific imagein the image collection.
 10. The method of claim 9, wherein making themulti-media image product comprises incorporating one of the pluralityof annotations stored on the data storage system.
 11. The method ofclaim 1, further comprising associating the annotation to the specificdigital image in accordance with the annotation assignment instruction,wherein the multi-media image product further includes the specificimage in association with the annotation received from the user.
 12. Themethod of claim 1, further comprising: receiving, from the user via thecomputer network, a request for a hard-copy product, wherein theannotation received from the user is a text annotation, and whereinmaking the multi-media image product comprises making the requestedhard-copy product in accordance with the first presentation format. 13.The method of claim 1, wherein the identifier is selected from the groupconsisting of: an email address, a URL, a URI, a text string, a barcode, a QR code, and a telephone number.